Learning design and assessment with e-tivities

This paper reported on the findings of research into innovation in e-learning design and assessment through the development and implementation of online learning activities (e-tivities). The focus of the study was on Carpe Diem as a process to enable academic course teams to seize 2 days to design and embed pedagogically appropriate e-tivities into their courses. The study also addressed the use of technology in the design of e-tivities and the level of tutor and learner engagement with them during course delivery. Six academic course teams representing three disciplines at four British universities took part in this 12-month study. Cognitive mapping was the main research methodology used. The results suggested that Carpe Diem is an effective and powerful team-based process to foster pedagogical change and innovation in learning design and assessment practices. The e-tivities designed during Carpe Diem were successfully used primarily for learning and formative assessment, and exceptionally for summative assessment. Web 2.0 tools, especially wikis, were employed to enable collaborative online learning and were prominent in the new designs. The tutors’ e-moderation skills were key to engage learners and thus capitalise on the benefits of e-tivities. Introduction and focus E-tivities are defined as ‘frameworks for enhancing active and participative online learning by individuals or groups’ (Salmon, 2002, p. 3). They can be used in a wide range of face-to-face, online and blended learning settings across disciplines and may be designed to make use of many learning technologies. E-tivities are low-cost, reusable, customisable and scalable. British Journal of Educational Technology (2009) doi:10.1111/j.1467-8535.2009.01013.x © 2009 The Authors. Journal compilation © 2009 Becta. Published by Blackwell Publishing, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA. Carpe Diem (Armellini & Jones, 2008; Salmon, Jones & Armellini, 2008) is a wellresearched, team-based intervention to promote innovation in learning design and assessment practices by academic course teams. The intervention includes a 2-day workshop in which teams design online learning activities (e-tivities) for effective and collaborative learning within their online, blended and face-to-face courses. This paper reports on the findings of the ADDER (Assessment and Disciplines: Developing E-tivities Research, www.le.ac.uk/beyonddistance/adder) project. As part of the project, academic course teams took part in Carpe Diem interventions to design and embed e-tivities in their courses for learning and assessment purposes. ADDER was funded by the UK Higher Education Academy and led by the Beyond Distance Research Alliance at the University of Leicester. The project compared and contrasted assessment practices that make use of e-tivities in three disciplines (media studies, psychology and inter-professional education, a discipline that aims to enable multiple health professionals to work together effectively) at four universities (De Montfort, Derby, Northampton and London South Bank) between October 2007 and September 2008. We report on the outcomes of this research, in which we: • Evaluate the effectiveness of the Carpe Diem intervention in terms of innovation into practice and change towards learner-centred, task-based learning design and assessment in the three disciplines at the four participating universities over 12 months. • Characterise the e-tivities produced by academic course teams during Carpe Diem and establish whether and how these e-tivities link to formative and summative assessment. • Identify tutors’ learning technology choices and their rationale for its use in e-tivity design. • Identify the factors influencing tutor and learner engagement with e-tivities during course delivery. • Explain how Carpe Diem helped academic course teams shift from practices based on the use of technology for delivery of course content to the development and management of tasks for collaborative knowledge construction and assessment. Learning design and assessment Learning design has been defined as ‘an application of a pedagogical model for a specific learning objective, target group and a specific context or knowledge’ (Conole & Oliver, 2006, p. 5). Recent literature on learning design points to the diverse challenges faced by academics in terms of pedagogy, technology and learning context (Beetham & Sharpe, 2007; Conole, Dyke, Oliver & Seale, 2004; Conole & Fill, 2005; San Diego et al, 2008). Many academics default to traditional practices, resulting in innovation in learning design hardly ever becoming a priority for them. Consequently, tutors often have limited awareness of what constitutes good practice in participative learning design and which learning technologies can be deployed in its implementation. Given the varied demands on their time and growing academic, administrative and pastoral responsibilities, lecturers need and welcome guidance on learning design leading to 2 British Journal of Educational Technology © 2009 The Authors. Journal compilation © 2009 Becta. pedagogical innovation (San Diego et al, 2008). This guidance has the potential to make their teaching more effective, engaging and rewarding. Assessment is central to learning and therefore key to learning design. The curriculum is defined by assessment (Ramsden, 1992), which shapes the student experience (Brown & Knight, 1994). Assessment ‘orients all aspects of learning behaviour’ (Gibbs, 2006, p. 23). Formative assessment can improve student learning and raise academic standards (Black & Wiliam, 1998; Yorke, 2003). Formative assessment practices empower students to develop skills to monitor, judge and manage their learning (Nicol, 2007). Students possess, according to Nicol, self-assessment and selfregulation skills. Effective formative assessment depends on students developing these skills (Nicol & Macfarlane-Dick, 2006). Tutors may build on this capacity rather than solely provide expert feedback themselves (Nicol, 2006). Learners are more likely to benefit from feedback that is timely, relevant and appropriate in terms of its content and how it is offered. These are crucial attributes of feedback, as they shape the ways in which learners engage with it. The quality of feedback has been found to be less important than the quality of learners’ engagement with the feedback (Gibbs; Nicol & Macfarlane-Dick). Armellini, Jones and Salmon (2007) found that assessment shapes the design of e-tivities and identified links between e-tivity design and assessment. This paper reports on the process of change in learning design and assessment implemented by course teams as a result of the introduction of e-tivities, the types of e-tivities produced by course teams and the technologies used, and the engagement of tutors and learners with those e-tivities during delivery. Carpe Diem Carpe Diem promotes and supports change in learning design and assessment, builds institutional capacity and fosters scalable pedagogical innovation (Armellini & Jones, 2008; Salmon et al, 2008). At the heart of this intervention is a 2-day workshop in which course teams, in collaboration with subject librarians and learning technologists, design e-tivities for effective e-learning and assessment within their online and face-to-face courses. The workshop is structured around six stages. On the first day, the team produces a blueprint (Stage 1) and storyboard for the course (Stage 2), identifying the purpose and main features of the e-tivities they will design. On the second day, participants build prototypes (Stage 3) and turn them into fully functional e-tivities (Stage 4) that they upload to their institutional virtual learning environment (VLE). A ‘reality checker’ (a student or staff member external to the Carpe Diem process) reviews the e-tivities and provides feedback from the user’s perspective. The team uses this feedback to adjust and improve the e-tivities (Stage 5). A clear action plan, including the next steps in the design of the course(s), is then agreed upon (Stage 6). Learning design and assessment with e-tivities 3 © 2009 The Authors. Journal compilation © 2009 Becta. At the end of the workshop, teams have a series of ‘reality-checked’ e-tivities running on their VLE, a storyboard showing the purpose and location of those e-tivities within the course design, and an action plan for further development. The workshop is preceded by an initial contact meeting between the facilitator and the course team for preparation and motivation, and is followed up by a meeting designed to plan for the embedding of the changes into the course (Figure 1). Carpe Diem differs from traditional staff development approaches in so far as it focuses on the learning design needs specific to an academic course team taking responsibility for a programme of study. Its outputs can be used by the course team immediately and can inform the development of other course components. Carpe Diem is not a ‘how to use my VLE’ workshop. While participants become more skilled in the use of a range of VLE features, they do so in the process of addressing a pedagogical design challenge that the technology may help them resolve. Learning technologists and subject librarians provide additional input and support during the intervention. In the ADDER project, the Carpe Diem intervention was used to enable disciplinespecific course teams to design and incorporate e-tivities into their courses, in a pedagogically sound way, for collaborative learning and assessment. Based on Salmon’s e-tivity framework (Salmon, 2002), a template was used in Carpe Diem workshops to Initial contact meeting with departmental course team Course team Follow-up meeting with course team All participants should have access to their institutional VLE. The workshop takes place on two consecutive days, normally from 10 to 4:30. All team members must attend on both days. The 2nd day of the workshop is run in a computer lab. Outputs of the 2 days: Purpose: embedding changes Activities: Two-day Carpe Diem workshop Who: A course team in a single discipline (4 to 20 participants) involved in a course redesign process, a subject librarian and a learning technologist. Observers may include staff developers.